Categorized document bases

ABSTRACT

A method of managing information comprises generating a categorized document base. Generating the document base comprises providing a pre-existing classification of things other than documents, providing a source collection of documents, and automatically assessing the documents using Information Retrieval techniques to assign at least some of the documents to one or more taxa of the classification. For each taxon in the classification one or more numerical scores are assigned, based at least in part on a composition, makeup or constitution of the documents assigned to the taxon of the categorized document base.

CROSS REFERENCE TO RELATED APPLICATIONS

U.S. Provisional Patent Application No. 60/730,423, filed Oct. 26, 2005,is incorporated herein by reference in its entirety. U.S. patentapplication Ser. No. 10/272,337, filed Oct. 16, 2002, is incorporatedherein by reference in its entirety. U.S. Provisional Patent ApplicationNo. 60/329,703, filed Oct. 16, 2001, is incorporated herein by referencein its entirety.

BACKGROUND

The present invention relates to statistical natural languageprocessing, including information retrieval, information extraction, andliterature-based discovery. More specifically, the present inventionrelates to obtaining a collection of documents that is classified orcategorized by a taxonomic system identified ex ante by the user, and tousing such a classified collection to obtain useful information.

Access to the right information is invaluable in the development of newideas and business opportunities, in supporting research andinvestigation on virtually any subject, and generally for making gooddecisions and alerting decision makers to conditions that requiredecision making.

A large portion of the important information that is needed for decisionmaking and that is stored electronically (and suitable for processing bydigital computers) is in the form of text in documents. It is generallyrecognized that the current state-of-the-art in statistical naturallanguage processing (which is the covering discipline and art foraccessing information in text documents) does not support fully adequateaccess to information in texts.

Categorized document collections, in which individual documents (ortexts) in a collection of documents are assigned to categories intaxonomic systems, are widely recognized and widely used for improvingeffective information access to collections of texts. Among othervalues, categorization serves to focus a decision maker's orinvestigator's attention on smaller subsets of larger collections,thereby facilitating search and retrieval. Also, the distribution ofdocuments across categories in a taxonomic system may itself be usefulinformation for decision making and investigation.

Examples of categorized document bases include library catalogs based onclassification schemes such as the Library of Congress classificationand the Dewey Decimal classification, and subject classifications suchas the United States Patent Classification and the International PatentClassification. Such classification schemes conventionally require ahuman being to examine a book or other document, and make a decision asto what class or classes to assign to the document.

It has been proposed to classify documents, for example, documentsgathered from the Internet, automatically by searching the text of thedocument for terms found in classification codes of an existing documentclassification and for terms found in an existing thesaurus to thatclassification. However, because the promoters of these proposals arelibrarians and library scientists, these proposals are typicallyconfined to generating a library-style classified index in which eachdocument is assigned to one or a few classification codes, and can beretrieved by searching the index under that code or one of those codes.Library indexes, even current computerized library indexes, aretypically limited to a search in a single index, or a search for theBoolean intersection of two or more unrelated indexes, (for example,classification AND author), returning a single list of “hits.”

There is therefore a continuing need for methods and systems that canprovide more information about documents than merely assigning thedocument to a class within a taxonomy.

SUMMARY

The present invention is directed to methods and apparatus for creatingcategorized document collections from collections of documents that arenot already so classified, including creating a new or distinctclassification of a source collection of documents that is alreadycategorized independently in some fashion or other.

In one embodiment of the invention, a pre-existing classification ofthings other than documents is provided. The classification comprisesclassification codes and definitions of the things belonging in eachtaxon of the classification. A collection of documents is provided. Thedocuments are automatically assessed using Information Retrieval (IR)techniques to assign the documents to one or more taxa of theclassification. For each taxon in the classification one or morenumerical scores are assigned, based at least in part on a composition,makeup or constitution of the documents assigned to the taxon of thecategorized document base.

In another embodiment of the invention, there are provided methods andsystems for generating a categorized document base, comprising providinga source collection of documents, automatically assessing the documentsusing Information Retrieval (IR) techniques to assign at least some ofthe documents to one or more first categories, and assigning for eachfirst category one or more numerical scores based at least in part on acomposition, makeup or constitution of the documents assigned to thecategory.

In a further embodiment of the invention, there are provided methods andsystems for providing at least first and second sets of categories,providing a source collection of documents, at least some of thedocuments assigned to one or more categories of each set of categories,and generating at least one of an array of documents and an array ofdata relating to documents, wherein the categories provide axes of thearray.

The system may be embodied at least in part using one or more computers,and another embodiment of the invention provides programs for causing acomputer to carry out the methods of the invention.

Many classification schemes are hierarchical, treelike structures, with“leaf” nodes, which are the lowest level, most specific, categories, andhigher-level nodes in the hierarchy that correspond to categoriescomposed from or including their subcategories down to the leaves. Bothleaf and higher-level categories are known as “taxa.” Related forms arealso recognized, including simple, non-hierarchical lists of taxa, inwhich case all taxa are leaf nodes, excepting possibly a root nodeidentifying the entire taxonomy. Alternatively, the graph topology of aclassification system may be a network (or even a more complex graphstructure), rather than the prototypical tree or simple list. Inaddition, a taxonomy or classification system may be faceted and attachmultiple classification systems to individual entities.

In general, a “taxon” (plural “taxa”) is any location within aclassification system to which an item can be assigned. In particular,in a hierarchical “tree” classification the number of levels ofclassification may vary from branch to branch, so that a leaf taxon onone branch may be level with a non-leaf taxon on another branch. An itemmay be assigned to a non-leaf taxon, for example, by inheritance from aleaf, so that a higher-level taxon explicitly includes all the items inits subcategories. Alternatively, the non-leaf taxon may benon-exhaustively sub-classified, so that there are leaves (orlower-level “twig” taxa) assignable to only some items within thenon-leaf taxon, which thus becomes the lowest-level classification foritems that do not have an assignable leaf.

In a further embodiment of the invention, the classified documents arethen searched according to one or more criteria different from theclassification, and an analysis of the number of documents in each taxonhaving the one or more criteria, or having a specific aspect of the oneor more criteria, is generated.

This analysis may be quantitative and may constitute or contribute tothe one or more assigned numerical scores. For example, the system mayreport the number of times a specified word or phrase occurs in adocument, either directly or as a functional transformation, optionallybased on other information, such as a fraction or percentage of thenumber of words or other measure of the size of the document, or of thetotal size of all the documents in the same taxon of the initialclassification.

The documents may be searched and analyzed according to three or moredifferent sets of criteria including the initial classification whereapplicable. The documents may be grouped according to some sets ofcriteria, and a report generated analyzing the documents in a groupaccording to the other sets of criteria.

In another embodiment of the invention, documents may be automaticallyclassified according to any pre-existing classification using IRtechniques. The documents may then be searched according to a pluralityof states of one or more other criteria, and an array of documents indifferent taxa of the classification and different states of the one ormore other criteria is generated.

The one or more other criteria may be provided by one or more otherclassifications, in which case the plurality of states comprisesassignment to taxa in the other classification. The results may then bepresented and/or manipulated in the form of an array, with each distinctclassification providing a dimension of the array. The one or more othercriteria may be, for example, a plurality of keyword searches, in whichcase the plurality of states may comprise positive matches, or scoresabove a threshold, for different keywords. The results may then bepresented and/or manipulated in the form of an array, with theclassification (or each distinct classification) providing a dimensionof the array, and the keyword searches (or each distinct set of keywordsearches) providing a dimension of the array.

Another aspect of the present invention provides a general method forcreating categorized document bases by using pre-existing taxonomies (orclassification schemes) as sources of search terms for informationretrieval techniques. Given a collection of documents and aclassification scheme, the present method teaches ways to generate thecategorization of documents in a collection in accordance with, and byactively using, a given classification scheme. While the requiredclassification scheme must exist prior to execution of the presentmethod, it may be created by users on an as-needed basis, e.g.,immediately prior to employment of the method.

In general, a “user” of a system according to the invention may includeowners or operators of the system, whether using the system for theirown benefit or for the benefit of others, and customers, who aretypically users of the information generated who commission an operatorto prepare a classification or analysis for a specific purpose. Where anoperator and a customer are involved, the functions and responsibilitiesof the “user” may be distributed between them according to the expertiseand interests of each.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

FIG. 1 is a schematic diagram of a scheme for generating a categorizeddocument base.

FIG. 2 is a schematic diagram similar to FIG. 1, illustrating a firstembodiment of the present invention.

FIG. 3 is a flowchart of an embodiment of the present invention.

FIG. 4 is a schematic diagram similar to FIG. 2, illustrating a furtherembodiment of the present invention.

FIG. 5 is a flowchart of another embodiment of the present invention.

DETAILED DESCRIPTION

Reference will now be made in detail to various embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

Referring to the drawings, and initially to FIG. 1, in creating ageneralized categorized document base (CDB), by some procedure or methoddocuments in an existing collection of source documents 10 are assignedto, or informationally associated (as in a record in a database table,or other electronic file) with particular categories or classifications12, 14, 16, 18, 20. This process of assignment results insub-collections of the documents from the original collection. Eachmember of a sub-collection is associated with a common category ortaxonomic entity (taxon). It is not necessary for every document to beassigned to some category, although that is possible. Where it isdesired for every document to be assigned to a category 12, 14, 16, 18,20, it may be necessary for the categories to include a “null” category(not shown) defined as the category of all documents not assigned to anyother category. It is not necessary that every assigned document beassigned to exactly one category, although that is possible. Manyclassification schemes permit a document to be assigned to two or moredifferent categories. In addition, in a hierarchical taxonomy a documentmay be assigned both to a fine category and to a coarse category ofwhich the fine category is a subset.

The specific categories 12, 14, 16, 18, 20 shown in FIG. 1, which are:shoes, ships, sealing wax, cabbages, and kings, are for purposes ofillustration only. The concept of a CDB can be applied to anycategorization scheme.

As an example, a user may wish to categorize a collection of documentsaccording to type of product (if any) that is relevant to the documentsin a collection. The user may be a “customer.” The customer is notnecessarily the owner or operator of the classification system, but maybe a person or organization wishing to use the classified documentcollection, who commissions the operator of the classification system toprepare a specific classified collection. Further, the user has aspecific classification system in mind for which the user wishes tocategorize a source collection of documents. For the sake of theexample, let that classification scheme be the United Nations StandardProducts and Services Code. With the present embodiment, this can bedone by starting with a product classification system (e.g., UNSPSC) andusing the system itself as an integral part of the process of creatingthe CDB. The present process uses information retrieval methods on anyexisting classification scheme and does not require specialized ruleformation for matching documents to categories.

Referring now to FIGS. 2 and 3, in one form of method according to thepresent invention, in Step 102 a source document collection 10 isidentified and access to the collection is obtained. Examples of sourcedocument collections include but are not limited to: documents posted onthe World Wide Web, documents retrieved from the World Wide Web by oneor more queries posed to a search engine, and documents retrieved from adocument collection by one or more queries posed to an InformationRetrieval System (IRS).

In Step 104, a classification scheme is identified. As shown in FIG. 2,the classification scheme comprises taxa 22, 24, 26, . . . 28, 30represented by the terms shoes, ships, sealing wax, . . . , cabbages,kings. The classification scheme, and taxa, shown in FIG. 2 are forpurposes of illustration only. The present process permits the use ofany articulated classification scheme or taxonomy, including, but notlimited to, schemes referred to by other terms of art such as“ontology,” “controlled vocabulary,” and “categorization scheme.”

In Step 106, search queries are generated from the taxa of theclassification system. The search queries may be simply keywords, orBoolean combinations of keywords, from the taxon definitions in theclassification system, to be used as search strings for a text search.In general, a “search string” or “search query” may be any input to anInformation Retrieval System (IRS) that provides the IRS with one ormore criteria for selecting some documents from a source collection,known as “hits,” and rejecting other documents from the sourcecollection. A query that results in all or none of the documents beingselected is logically possible, but usually unhelpful. Where the queryis a word, phrase, or Boolean combination of words, searching maycomprise searching for the exact search terms, words and phrasesspecified, similar words and phrases, and patterns of search terms,words, and phrases. A particular query, when applied by a particularIRS, may result in additional states other than “hit” or “not hit.”

In Step 108, an information retrieval system is provided. The details ofthe information retrieval system depend on the source documents and thesearch queries. For example, in the example of a search of the WorldWide Web using keywords from the taxon definitions, the informationretrieval system may be essentially an available World Wide Web searchengine.

In Step 110, the search is conducted, and appropriate informationretrieval methods, which may be methods established in the art, are usedto identify “hits,” that is to say, matching documents, in thecollection of source documents. For example, hits might be defined asdocuments containing the taxa terms. In FIG. 2 these terms are shoes,ships, sealing wax, . . . , cabbages, kings. Thus, the categorizeddocuments for the category of shoes might consist of all the documentsin the source documents collection containing the word shoes, andsimilarly for the other taxa.

In Step 112, the discovered associations between taxa and their hits arerecorded in electronic form for convenient retrieval and subsequent use.For example, the information may be recorded in a relational database orin a variety of purpose-built file formats. The discovered associationsdefine the sub-collections of documents 12, 14, 16, . . . 18, 20. It isnot always necessary, though it is sometimes possible, to generate setsof copies of the documents in each sub-collection. Depending on thenature of the source documents, it may be preferred to generate only alisting that identifies the documents and their locations or, asdescribed below, some intermediate amount of information.

A wide variety of procedures are available from the field of informationretrieval for using search terms to obtain hits from a collection ofsource documents in Step 110. Any of these methods may be employed inthe context of the present embodiment, by using taxa descriptions(“shoes,” “ships” and so on in the example) or expansions thereof (e.g.,synonyms, hypernyms, hyponyms) as search terms.

The method of FIGS. 2 and 3 applies as well to iconographic taxa, suchas images or patterns, which may be matched to non-textual documents orparts of documents. However, classification of images is not welladvanced, and usually relies on classifying a verbal description of theimage, or of the object depicted in the image, rather than the imageitself. Because unrelated objects can have very similar appearances, andbecause the same object or objects of the same description can have verydifferent appearances, the construction of a thesaurus of imagedescriptions is much harder than the construction of a thesaurus ofwords, and classifications of images are typically far less reliablethan classifications of text documents.

Matching of taxa to documents (using information retrieval techniques,or other techniques appropriate for iconographic taxa) does not have tobe exact. For example, there are known information retrieval techniquesthat will rank an entire collection of documents based on a singlesearch term, whether or not it occurs in all of the documents. It ispossible, and appropriate, that under such a ranking a document mighthave a high relevance score and not contain the exact search term. Inany event, using a taxon or a related expression as a search term, thereare information retrieval methods that will rank a document collectionby relevance and/or return a relevance score for a number of documents.These scores may be used to determine whether a document is assigned toa specific taxon.

For example, if the search term is “shoes” (and if matching isrestricted to exact matches to the text string “shoes”), and if theinformation retrieval method employed gave ranking scores or relevancescores to some or all of the documents in the source collection, thenthe documents assigned the category of “shoes” could be those documentswith a specified, sufficiently high score, for example, the top 500documents, or all documents mentioning “shoes” at least 20 times.Setting a high threshold for relevance or ranking typically results incollections of documents with high relevance, but may result indocuments of moderate relevance being excluded, and may result in asubstantial number of documents from the original collection beingdisregarded or assigned to a “null” or “null with respect to matching”category even when the taxonomy is in principle exhaustive.

Referring now to FIG. 4, a further embodiment of the present methoddiffers from the embodiment shown in FIG. 2 only in that the taxondescriptors are augmented with additional information when used in Step106 as input to the procedure for creating queries. One example ofaugmentation is known in the information retrieval literature as queryexpansion. Under a query expansion regime, taxon descriptors would beaugmented with related terms, e.g., to “shoes” the search terms “shoe,”“boot,” “footwear” and so on might be added and this augmented listwould be used by the information retrieval engine to identify hits. Thepre-existing taxonomy used to define the taxa may have a pre-existingthesaurus of approximate synonyms to the taxon descriptors.

Techniques for query expansion and for other forms of query augmentationare well known in the field of information retrieval. The presentprocesses employ them innovatively, for augmenting categorizationdescriptions for the purpose of creating taxonomically generatedcategorized document bases.

As a second example, taxonomic descriptions may be augmented by entiredocuments, following the concept of “exemplary documents.” Thus, forexample, the taxa may be used to search for highly relevant documents,and then the most relevant documents may be used to augment the taxarepresentations for the purpose of directing an information retrievalsystem to find documents (“hits”) fitting the taxa. Also, manyclassification schemes, particularly thesauruses, are attended bytextual information elaborating on and clarifying the meanings of thevarious taxa descriptors. The term “scope note” is often used in thissense. The text of the scope notes may then be used in augmenting thetaxon descriptions.

In a further example of a method according to the embodiment of theinvention shown in FIG. 3, in Step 102 a user wishing or intending toobtain a categorized document base (or collection) identifies a documentbase (or collection) of source documents. Examples of possible sourcesfor the source document collection are identified elsewhere in thisspecification. Specifically, the collection of documents may include apre-existing search database, for example, the United States Patent andTrademark Office (USPTO) patent and application database, privatecollections, government documents, online databases and collectionsassociated with search engines, such as Google, Dialog, Medline,Lexis/Nexis, etc. The collection of documents chosen depends on theuser's intended use of the categorized document base and the availabletaxonomies and search engines to effect the classification. It should benoted that the choice of the collection of documents is not limited tochoosing one collection; multiple collections of documents may beutilized. If the source document base or collection is a proprietarycollection, or includes proprietary documents, the user obtains theappropriate permissions.

The collection of documents may be a collection that was generated by aprocess according to the present invention. Embodiments of the presentinvention produce a number of files that contain hits from pastsearches. These past search files may be used as starting collections toconduct more narrow searches.

Once the user has identified and secured access to a collection ofsource documents, a classification scheme, or taxonomy, for generatingqueries is retrieved by the user, in Step 104. As discussed above, the“classification schemes” or “taxonomies” are methods of categorizinginformation using a predetermined set of criteria.

An exemplary pre-existing classification that may be used in this stepis the Library of Congress (LoC) classification scheme.

In order to conduct the search, in Step 106 one or more queries aregenerated. In one example in the basic Taxonomically GeneratedClassified Document Base (TGCDB) case, the queries are search stringsgenerated by simply using n-grams that describe the taxa of theclassification scheme. An “n-gram” is defined as a sub-sequence of nitems from a given sequence, and in text processing is then usually aphrase of n consecutive words. Each query is constructed so as to beassociated with a taxon in the classification scheme. In some cases, asingle query may be associated with more than one taxon, if the query isso framed that a more informative result than a simple “match” or “nomatch” can be obtained.

Based in part on the collection(s) of documents that have been chosen,in Step 108 an appropriate Information Retrieval System (IRS) isacquired. The IRS will be the query and retrieval system that actuallyidentifies hits (documents) associated with the queries that have beengenerated in Step 106. Preferably, a generally available IRS, which maybe an already known IRS, is chosen, although an IRS may be utilized thathas been designed to conduct a search in accordance with an embodimentof the present invention. The IRS may be a proprietary system forproprietary collections, a standard search engine for Internetdocuments, a commercial information retrieval engine, or a purpose-builtsystem for example. Especially where the collection identified in Step102 is a combination of collections with materially differentcharacteristics, two or more different IRSs may be selected. Forexample, where part of the documentation forming the source collectionis in a highly structured proprietary database and part of thedocumentation is on the World Wide Web, a specific proprietary IRS maybe used for the proprietary database, and a standard search engine maybe used for the Internet documents.

In Step 110, the queries identified in Step 106 are input in the IRS(s)chosen in Step 108, and a search is conducted through the sourcecollection to find all hits that match the query, or match the query toa sufficient degree. A “hit” is defined as a record (document, ordocument ID) returned by an IRS using any of a variety of informationretrieval procedures when the IRS matches the document, or the documentto which the ID relates, to the query or one of the queries. The matchmay require a score for the quality of matching.

In Step 112, a record is created and stored for each hit, therebylinking each hit (document) with a taxon in the classification scheme.After determining how the hits are to be associated with theclassification scheme taxa, mapping of the hits to the appropriate taxais performed, building a mapping table (index table). An exemplarymapping table is illustrated in Table 1.

TABLE 1 Classification Scheme Taxon File ID UNSPSC Birds and fowl1-6417153.html UNSPSC Live chickens 1-6417153.html UNSPSC Live ducks9-6380149.html UNSPSC Live turkeys 10-6379753.html UNSPSC Live geese14-6360374.html UNSPSC Live geese 3-6416624.html UNSPSC Live pheasants2-6416633.html UNSPSC Live fish 10-6379753.html UNSPSC Live salmon2-6416633.html UNSPSC Live trout 12-6376045.html UNSPSC Live tilapia9-6380149.html UNSPSC Live tilapia 12-6376045.html

The mapping table shown in Table 1 comprises, for each hit, thetaxonomy, the classification node (taxon) ID and a file ID. In theexample, the taxa descriptors themselves, which are used as inputs tothe IRS, also serve as node IDs. It is of course possible to haveseparate representation systems for node IDs and n-grams that describenodes. The file ID identifies a matched document. In this example, thematched document is the actual hit, and where more than one document isassigned to a single taxon the mapping table contains a separate row foreach document. Alternatively, the mapping table may have one row pertaxon, and the identified document may then be a second-level tablegenerated in Step 112 listing the hits matched to the taxon in question.

Any other information may be included in the mapping table, depending onthe information to be provided. Where users may choose among availableIRSs in Step 108 and/or may select which retrieval algorithms are to beemployed within a particular IRS in Step 110, the mapping table mayspecify which IRS(s) and/or algorithm(s) were used to generate each hit.

In the embodiment illustrated by Table 1, each hit is given an ID, hasits text processed for convenient display, and is then written to aunique HTML file (in a preferred embodiment), suitable for subsequentviewing through a standard web browser.

The mapping table is stored and utilized in creating classification nodefiles. A “classification node file” is a file, record, or other way ofrecording information that comprises information, including statisticalinformation, regarding the hits associated with each node or taxon inthe classification scheme. This information includes at least the actualhits that were retrieved and associated with the node, and in ahierarchical scheme the number of files associated with the direct andindirect children of the node or the node itself, and may include anyother information as desired. The classification node files allow theuser to navigate through the retrieved information such that anindividual would be able to see how many documents are associated withsome classification and view the hits that are linked to theclassification node. In a hierarchical system, each parent node islinked to its respective children, and so on.

In an alternative embodiment, information included in the classificationnode files can be displayed in an electronic tree, allowing the user tonavigate through the retrieved hits faster since the user is able to seethe tree and what each node means, as well as see the statisticsregarding what was found and actually view the hits under particularclassifications.

In an alternative embodiment, the IRS produces a numerical score forsome or all of the documents in the source collection, which scoreestimates the relevance or degree of fit of the document with the query.In this case, a hit may be any document receiving a score, or anydocument receiving more than a threshold score, and the mapping table inTable 1 may then include an additional column of information, with theheading of “Score,” and the data cells in that column could provide afield storing the match or relevance score of the document identified inthe “File ID” column with respect to the taxon from which the query wasderived.

Each hit record in, or linked from, the mapping table may include theentire document that contains a match to the query, or may containinformation about the original document, such as the title, author(s),ISSN, abstract, etc. The type of information included with the hit isdependent on the user's ability and permissions to access theinformation in the IRS and the information provided by the IRS. SomeIRSs, especially dedicated search engines for proprietary databases, donot have access to entire documents and only provide citations ofrelevant documents or abstracts thereof. Accordingly, the results ofstring searches on the chosen IRS(s) may result in one or more filesthat include a combination of document descriptions and full-textdocuments that are related to the identified queries. Alternatively,some or all of the hits may contain only a link or other reference tothe original location of the document.

In an alternative embodiment, in Step 114 the user may filter the hitsretrieved, by reviewing the hits and determining whether the hit isreasonable in view of the purposes for which the categorized documentbase (CDB) is being constructed or in view of predetermined parametersset for a given search, e.g., industries that should not be considered,types or formats of documents that should not be considered.

For example, in Table 1, only one of the documents identified asrelating to specific species of bird was also identified as relating to“birds and fowl.” The number of hits on “birds and fowl” may depend onwhether “birds and fowl” is treated as a 3-gram for the exact phrase, asa Boolean “birds AND fowl,” or a Boolean “birds OR fowl.” If the resultsare not what the user desired, it may be appropriate to reformat thesearch string or other query to compel a different interpretation of“birds and fowl.” In addition, or in the alternative, it may beappropriate to specify within the taxonomy whether or not the generictaxon “birds and fowl” includes all documents that are assigned tospecific species of bird.

In addition, the only document assigned to two different species of fishis 12-6376045.html, which is assigned to both “tilapia” and “trout.”This may suggest that there has been an alphabetical grouping of fishwithin the source documents that can be used or that should becompensated for.

In Step 116, the user decides whether it is useful to reclassify thesource document collection, for example, because the review in Step 114has led to a material change in the taxonomy or a material refinement ofsome or all of the queries, or has led the user to realize that adifferent IRS would be more effective. If the user deems it useful toconduct a new classification, the procedure continues by looping back toStep 104, Step 106, or Step 108.

If the user is content not to reclassify the document collection, theprocedure halts (is either terminated or suspended).

After the hits are identified and mapped to the taxa in Step 112, thesystem may prepare a report on the documents in each category, whichprovides, for example, available bibliographic data in a standard form,an abstract if one was provided at source, cross-references to otherhits in the same or closely related taxa, an analysis of frequent orunusually frequent words in the text of the hit, or any otherinformation that can be automatically retrieved or generated from thesource document and that will or could be useful to a personinterpreting the results in Step 114.

Referring now to FIG. 4, a further embodiment of the invention uses anaugmented TGCDB (Taxonomically Generated Classified Document Base)procedure. In one basic embodiment of the procedure described withreference to FIGS. 2 and 3, search strings are created in Step 106 byusing the literal n-gram form of the taxon descriptors of a chosenclassification scheme. Those search strings are then fed sequentially toan IRS and hits are identified.

However, there are a variety of known ways to augment an n-gram to beused for purposes of searching with an IRS. Many of these ways are formsof query expansion, that is to say, of adding search terms to a user'sexisting search terms, or query (here, the n-grams constituting thetaxon descriptors of the chosen classification system). The augmentationis typically intended to improve precision and/or recall, by finding“hits” that do not match the literal n-gram.

The additional terms may be taken from a thesaurus. For example a searchfor “car” may be expanded to: “car OR auto OR automobile.” Where the IRSdoes not provide stemming, automatic truncation, or support forinflected forms, the search for “car” may also be expanded to: “cars ORautos OR automobiles.” Users, or others, can add additional terms oreven sentences and paragraphs. The additional information can be addedeither manually or automatically. The additional terms may also be takenfrom documents that have been specified as being highly relevant orexemplary in the sense described above. (This kind of expansion is thebasis for the “more like this” feature of some search engines.) Further,the extra terms can have positive or negative weights. That is, theadded terms (or n-grams) may come with instructions to the IRS thatencourage it to find documents containing the terms or matching to theterms; similarly the IRS may be instructed to avoid returning or scoringhighly an added search n-gram.

In addition to query expansion, there are other ways of employingaugmenting information. Principal among these in the context of thepresent embodiment is the association of entire documents, ranging inlength from short paragraphs to substantial reports, with the taxa of aclassification scheme. This eventuality affords use of additionalInformation Retrieval algorithms for search and relevancy determination.

A user's decision to use augmenting information may impact a number ofsteps in the procedure. Different classification systems may servesimilar purposes, but some may have more or better-quality augmentinginformation than others. Some classification systems, in fact, areoriginally constructed with augmenting information attached to eachtaxon. This information is often referred to as “scope notes.” Forexample, The UNESCO Thesaurus also includes scope notes (SN) whichexplain the meaning and application of terms, and French (FR) andSpanish (SP) equivalents of English preferred terms. In an alternatepreferred embodiment, users may choose which of the availableaugmentation information for a classification system is to be used inconstructing search terms and queries.

A user's choice of IRS in Step 108 may be affected by whichclassification system is chosen, which augmenting information is to beused, and which search methods the user prefers to use. As in the basicTGCDB procedure, users may choose among available IRS(s), and within aparticular IRS may select which retrieval algorithms are to be employed.

In alternate preferred embodiments, the mapping of document hits to taxain Step 112 of either the basic TGCDB procedure or the augmented TGCDBprocedure may itself be augmented in a number of ways using InformationExtraction techniques.

Information Extraction (abbreviated IE) is the automated analysis ofunrestricted text in order to extract information about pre-specifiedtypes of events, entities or relationships. In various embodiments ofthe present invention, IE methods may be applied either to the documentsin the collection of source documents identified in Step 102 of eitherthe basic or augmented procedure or to the “hits” identified by the IRSin Step 110 of either procedure. IE methods may be applied for suchpurposes as extracting “sponsor” links or other information present inWeb pages returned by Internet search engines, and extracting otheruseful facts and data items from documents, as in, for example,extracting date, assignee, title, etc. information from patentdocuments. In the same spirit, other document information may beobtained and recorded, such as the length or number of words in adocument, its format, whether it has images, etc. In alternate preferredembodiments IE methods are applied, information is extracted and, inStep 112 of either procedure, the information is mapped to the documentIDs.

Referring now to FIG. 5, in one method of interpreting classified data,a set of documents classified according to a taxonomy is provided. Theclassified documents are preferably, but not necessarily, the outputfrom Step 112 of any of the processes and systems described withreference to FIGS. 2 to 4 above. Alternatively, the following steps ofFIG. 5 may be carried out using one or more CDBs independently createdby an arbitrary method.

The documents are searched according to one or more queries or searchcriteria different from those used to classify the documents accordingto the taxonomy. An array is then generated in which each cellcorresponds to documents that are assigned to a specific taxon in theinitial classification and that match a specific query in the furthersearch.

The different queries may be search strings or other queries derivedfrom taxa of a different taxonomy. Where the source document collectionhas already been separately classified under both taxonomies, and thedocuments themselves are clearly identified in the mapping tables fromboth classifications, the further search may be carried out byidentifying which two taxa each document is assigned to in the twomapping tables, and assigning that document to the cell of the arraythat corresponds to that pair of taxa (or each of those pairs, where oneor both taxonomies allows multiple classifications).

Preferably, however the second search is based on a plurality ofkeywords that do not form a comprehensive taxonomy.

For example, the present embodiment includes a search method and systemfor matching a given commodity with known and unknown applications,uses, markets and/or industries. The term “commodity,” as used herein,is construed broadly to include, but not be limited to, a technology,methodology, an idea, an invention, a product and services, which may beactual, envisioned, or even hypothetical. A commodity's properties anddescriptors, which may include not only known properties but alsoinferred, hypothesized, or conjectured properties, are then used as thebasis for the queries for the further search. Documents that areassociated with and are about these properties and descriptors are thenmapped by the array to a classification scheme of interest, which maysuggest hypotheses regarding new properties or applications of thecommodity. In particular, if a commodity's known properties areassociated with additional properties that fall into a certain category,e.g., taxon A, then the definition of taxon A can be used to suggest newproperties of the commodity. For example, if antiangiogenesis is aproperty associated with thalidomide, and many of the documentsretrieved using the Information Retrieval techniques are not aboutthalidomide, but they are about antianiogenesis, and the documentsregarding antianiogenesis may be classified as documents about cancertreatments, then there is a presumptive connection between thalidomideand cancer treatments.

In more detail, in Step 202, therefore, a customer or other user wishingor intending to obtain information regarding a certain commodity, asdefined above, identifies a commodity to be matched with one or morenovel uses, applications, markets or industries. For purposes of thisdisclosure, “markets” refers to connections between ideas andcommodities that may or may not have been previously considered.

Once a commodity has been identified, in Step 204, known properties,attributes, descriptive terms, etc. are developed for the identifiedcommodity. The known properties and descriptive terms may include, butare not limited to, known characteristics of the commodity, knownindustries associated with the commodity, attributes of the commodity,such as its texture, use, associations, designations and the like. Thesedescriptive words and properties may be derived from dictionaries andthesauruses, for example, or generated by an automated means, such as anelectronic thesaurus or electronic means. It should be noted that theproperties and descriptive words may be unlimited and may go beyondwords that specifically describe the identified commodity, which can begenerated, e.g., via a process of structured interviews, careful readingof known source materials, and feedback from experts. Returning to theearlier example, for instance, properties relating to the termthalidomide may be very diverse, including, e.g., causing relaxation,teratogenesis (causing birth defects), and antiangiogenesis (inhibitingformation of blood vessels).

Once the properties have been generated, a relevant classifiedcollection of documents to search is identified in Step 206. Thecollection of documents, and the taxonomy by which those documents areclassified, may include any of the collections of documents mentionedelsewhere in this application. The collection of documents chosendepends on the commodity identified and the generated properties. Theclassified collection may be generated using the process of FIG. 3 whenit is required in Step 206. Alternatively, the operator of the systemmay maintain a library of classified collections of documents formed byclassifying well-known source collections according to recognizedtaxonomies, and the customer may then choose from such a library,avoiding the effort and delay that would be involved in generating aclassified collection to the customer's order. In choosing theappropriate classified collection, the customer may make use of variouskinds of information about the source collections, the classificationschemes, and the classified collections that are available. Suchinformation may include notes on intended uses and origin from thecreators of the classification scheme, and/or user reviews andevaluations of a classification scheme.

Based in part on the collection of documents that have been chosen, anInformation Retrieval System (IRS) is acquired or selected in Step 208.The IRS may be, for example, any of the IRSs mentioned elsewhere in thisapplication.

In order to conduct the search using the IRS(s) chosen in Step 208,queries must be generated in Step 210. The queries may be searchstrings. Search strings are then preferably generated by combining oneor more of the properties that were generated in Step 204. Thecombinations may be arranged using Boolean connectors, e.g., AND, OR,including connectors that are not strictly Boolean, but are generallyreferred to as such, for example, “in the same sentence as,” etc. Thesearch may be broadened, either in forming the search strings or inconfiguring the IRS, to include, for example, pattern matching for wordsor phrases similar to those in the search string, or associationsearching using thesauruses, scope notes, and so on. Although a Booleansearch is preferred, any type of search may be utilized that takes thegenerated properties and forms a search string. The generated searchstrings or other queries are used to search the selected collection(s)of documents.

In FIG. 5, Step 210 of creating the queries is shown after Step 208 ofchoosing the IRS, whereas in FIG. 3, Step 106 of creating the queries isshown before Step 108 of choosing the IRS. Either order may be followed,or the steps may overlap or interact. In particular, a customer,operator, or other user may choose specific queries that express thecustomer's interest cogently, and select an IRS that can apply thosequeries quickly and effectively, or the user may select an IRS that cansearch the classified collection effectively, and frame queries that theselected IRS can apply efficiently.

In an alternative embodiment, a method is utilized which automaticallygenerates the queries. This alternative method takes the generatedproperties from Step 204 and creates a plurality of possiblecombinations of the properties alone and together with other properties.Once all possible combinations have been generated, a search isconducted for each combination using the IRS(s) chosen in Step 208. Thenumber of hits found for each of the combinations is tallied andutilized to determine the best queries based on predetermined criteria,for example, that the number of documents found must be within apredetermined numerical range. The numerical range may have both upperand lower limits. The combination queries that meet the predeterminedcriteria are utilized as the queries for the substantive search in Step212.

In Step 212, the queries found in Step 210 are input in the IRS(s)chosen in Step 208, and a search is conducted to find all hits that havethe query located therein. A “hit” is defined as a record returned by anIRS, broadly construed. The hits that are found are identified, countedand saved.

Where the documents referenced, directly or indirectly, in the mappingtable for the original classification and stored in association with themapping table contain the entire text hits from the search on which theoriginal classification was based, the search in Step 212 may beconducted on those documents. Where those documents contain less thanthe whole of the hits, for example, contain only bibliographic datawhere at least an abstract is available, contain only an abstract wherea fuller text is available, or contain only pointers to the originaldocuments, the search in Step 212 may involve tracing and searching theoriginal documents.

In Step 214, the results of string searches using the chosen IRS(s) onthe classified documents are presented in an array of files that mayinclude a combination of document descriptions and full-text documentsthat are related to the identified commodity's properties and attributeschosen in Step 204. Positions along the axes of the array typicallyrepresent a taxon within the initial classification in one dimension,and the different queries of Step 210 in the other dimension.

The array generated in Step 214 may be embodied in a user interface,which allows users ready visual access to information in the form of atable, or may be embodied as non-visualized data structures in acomputer program and, as such, searched and otherwise manipulated by acomputer program.

In schematic form, the array generated in Steps 206 to 214, which is a“classification-query” mode of generation, may be represented as inTable 2.

TABLE 2 Queries: Taxa: Q₁ Q₂ . . . Q_(n) T₁ s(1, 1) s(1, 2) . . . s(1,n) T₂ s(2, 1) s(2, 2) . . . s(2, n) . . . . . . . . . . . . . . . T_(m)s(m, 1) s(m, 2) . . . s(m, n)

In a preferred embodiment the table is produced as follows. First, acategorized document base (CDB) is created using m taxa, T₁, . . . ,T_(m), based on a source collection of documents, C. In consequencedocuments in C are associated with each of the taxa T_(i). If some taxahave no associated documents, the associated scores for the row of thetable are nulls, 0s, some other indicator that can be correctly handledin the specific implementation. The CDB is provided in Step 206.

Second, in Step 210 a number n of queries Q_(j) are identified. Thequeries may be, for example, n-grams representing keywords or phrases,or collections of n-grams combined with operators from an extendedBoolean matching system, or even entire documents.

Third, using IR methods or other suitable methods in Step 212, eachquery is assessed with respect to the documents categorized by eachtaxon. In effect, m document collections are separately queried n times(once for each query) for a total of m×n query processing events. Ineach case (of applying a query, Q_(j), to the documents associated witha taxon, T_(i)) one or more statistics, s(i, j), are obtained forcharacterizing the cell of the table (i.e., the document collectionassociated with T_(i) as queried by Q_(j)). Statistics which may beobtained in this fashion include, but are not limited to:

1. The number of documents (in a taxon) for which the query exactlymatches or has a certain degree of match to a string or string pattern(for example, a query string “organic compounds?” might indicate apatterned query which matches to either “organic compound” or “organiccompounds”);

2. The total number of times the query is matched in the taxon documents(possibly many times in a given document);

3. The proportion of documents in the taxon which match the query, orthat match above a certain threshold, for example, for the number ofhits in the document;

4. The percentage of the C documents (documents in the sourcecollection) that match to both taxon T_(i) and query Q_(j).

5. An IR matching score (e.g., number of documents hit, relevance score,rank, probability of relevance, etc.) for cell (T_(i), Q_(j)), that is,for the IR match of Q_(j) to T_(i) or the IR match of T_(i) to Q_(j).

6. The number of times the query is matched in the highest-matchingdocument in the taxon T_(i), or in the query Q_(j); and

7. Functional transformations and combinations (possibly including otherinformation) of the above, including but not limited to lift andreceiver operating characteristic (ROC) curve calculations (involvingmultiple taxa), smoothing functions (e.g., converting integer scores toreal numbers), and other calculations for such purposes as exploratorydata analysis (e.g., means, medians, box plots, etc.).

Extensive diagnostic reports may be produced for a cell. One such reportis a table of statistics on co-occurrence of terms in the documentsassociated with the cell.

In addition, scores may be reported as complex data structures. Forexample the distribution of matches in a category may be presented insome form (such as a box plot or an array of values). In one preferredembodiment, a number of scores (simple and/or complex) are madeavailable to the user or to a program executing on behalf of the user.

In an alternative embodiment, marginals are determined and displayed fortaxa, queries, or both, as illustrated schematically in Table 3.

TABLE 3 Queries: Taxa: Q₁ Q₂ . . . Q_(n) Taxa Marginals: T₁ s(1, 1) s(1,2) . . . s(1, n) m(T, 1) T₂ s(2, 1) s(2, 2) . . . s(2, n) m(T, 2) . . .. . . . . . . . . . . . . . . T_(m) s(m, 1) s(m, 2) . . . s(m, n) m(T,m) Query Marginals: m(Q, 1) m(Q, 2) . . . m(Q, n)

Like the scores, s(i, j) shown in Table 2, the marginals, m(T, i) andm(Q, j), may be present either as simple scalar numbers or textliterals, or as complex data structures, containing multiple scalars,literals, or combinations thereof. The marginals are summary ordiagnostic statistics for the associated taxon or query, and may come inmany forms. For example, taxa marginals may report the number ofdocuments in a taxon, the percentage of the relevant collection that isassociated with the taxon, the number of documents in the taxon that arenot classified with other taxa, and so on. For further example, querymarginals may report the number of documents in the relevant collectionthat matched or had a certain degree of match to the query, the numberof taxa in which there were documents that matched the query or thatmatched to a certain degree or that numbered above a threshold, and soon. As in the example of Table 2, extensive diagnostic reports may beproduced for a row or column. Marginal scores are especially suitablefor statistics that are cumulative over a column or row, or that analyzevariations over a column or row.

Users, and programs working for users, may manipulate the datastructures reporting results (Tables 2 and 3) and may extract reports.Such manipulations include but are not limited to sorting the score datas(i, j) into a desired order by row and/or by column (and their analogsif more than 2 dimensions are present), and in other ways rearrangingthe order of the rows and/or columns; functionally transforming thescores. Examples of extracted reports include but are not limited tovisually highlighting scores that meet a specified condition (e.g.,level above a threshold) and charting (plotting, displaying graphically)the scores (or some portion of the scores) for visual inspection.Programs working for users may inspect the scores, rearrange the rowsand columns (and their analogs if more than 2 dimensions are present),and search for trends, associations and other forms of patterns amongthe scores. Users may also manipulate the data structures reportingresults (Tables 2 and 3) in order to display the contained informationin various different ways, known to those skilled in the art ofinformation display.

The structures shown in Tables 2 and 3 may be generalized to 2 or moreclassification schemes and/or 2 or more distinct query sets, giving a“table” or array with 3 or more dimensions.

A further example, a “classification-classification mode,” is shown inTable 4, in which both axes represent distinct taxa.

TABLE 4 Taxa 2: Taxa 1: T₁ ² T₂ ² . . . T_(n) ² T₁ ¹ s(1, 1) s(1, 2) . .. s(1, n) T₂ ¹ s(2, 1) s(2, 2) . . . s(2, n) . . . . . . . . . . . . . .. T_(m) ¹ s(m, 1) s(m, 2) . . . s(m, n)

In a preferred embodiment the table is produced as follows. First, acategorized document base (CDB) is created using m taxa of a firstclassification scheme, “classification scheme 1,” based on a sourcecollection, C of documents, D. In consequence documents D in C areassociated with each of the taxa in classification scheme 1. Second, acategorized document base (CDB) is created using n taxa of a secondclassification scheme, “classification scheme 2,” based on the samesource collection C of documents D. In consequence, documents in C areassociated with each of the taxa in classification scheme 2. Aspreviously described, some taxa may have no associated documents, inwhich case the associated scores for the associated row or column of thetable are nulls, 0s, some other indicator as suits the implementation.Third, the documents in C that are matched by, or classified in, bothtaxon T¹ _(i) and taxon T² _(j) are counted as in the joint taxon (T¹_(i), T² _(j)) for which the score s(i, j) reports summary informationand statistics. Statistics which may be obtained in this fashioninclude, but are not limited to, statistics analogous to those mentionedfor the classification-query mode of Table 2, with the joint taxon (T¹_(i), T² _(j)) corresponding to the cell (T_(i), Q_(j)).

In an alternate embodiment, each taxonomy is used to classify a documentcollection. It is allowed (but not required) that each classificationscheme may be applied to categorize a different collection of documents.Thus, taxonomy 1, T¹ is applied to document collection C¹, taxonomy 2,T² is applied to document collection C², . . . , and taxonomy t, T^(t)is applied to document collection C^(t), where the various collections,C^(i), may be distinct. Then, the documents classified by an individualtaxon x of classification scheme y and collection z, named by D_(x,y,z),are combined to form and be treated as a single document for thepurposes of deriving association scores, which are obtained by using anIR association matching function, preferably one that returns a measureof similarity between two documents. Thus, a generic entry in the table,s(i, j), represents a match score between the combined documents intaxon i of classification scheme T¹ and the combined documents in taxonj of classification scheme T² (mutatis mutandis for tables or arrays ofmore than two dimensions). As an example, the table has 2 dimensions,with classification schemes T¹ and T² being used for both dimensions ona common collection of documents. In this special case, the scores mayrepresent measures of similarity between documents associated with twotaxa. Among other purposes, this information is useful and valuable tousers who wish to diagnose the discriminative success of the appliedclassification scheme.

In addition, scores may be reported as complex data structures. Forexample the distribution of matches in a category may be presented insome form (such as a box plot or an array of values). In one preferredembodiment, a number of scores (simple and/or complex) are madeavailable to the user or to a program executing on behalf of the user.

In other embodiments, relationships other than AND (logical conjunction,^) are supported in associating the classifications. For example, thedocuments in C that are matched by, or classified in, taxon T¹ _(i) @taxon T² _(j) may be counted as in the joint taxon (T¹ _(i), T² _(j))for which the score s(i, j) reports summary information and statistics,where @ may be any of a variety of relationships, including but notlimited to: logical conjunction (AND, as above), logical disjunction(OR), logical negation or complement (−), as well as other relationshipssuch as “appears next to” and “appears before.”

The remarks on further uses with reference to Table 3 apply mutatismutandis to Table 4 uses as well.

A further embodiment, illustrated schematically in Table 5, is a“query-query mode,” in which both axes of the table are provided by setsof queries.

TABLE 5 Queries 2: Queries 1: Q₁ ² Q₂ ² . . . Q_(n) ² Q₁ ¹ s(1, 1) s(1,2) . . . s(1, n) Q₂ ¹ s(2, 1) s(2, 2) . . . s(2, n) . . . . . . . . . .. . . . . Q_(m) ¹ s(m, 1) s(m, 2) . . . s(m, n)

The table may be produced analogously to Tables 2 and 4, once a sourcecollection of documents is identified, C. From this point, queries 1 and2 may be viewed as classification schemes, and so the case reducesmutatis mutandis to the case shown in Tables 2 and 4. Marginals may beadded as appropriate to the rows and/or the columns of Table 4 or Table5, as illustrated by the difference between Table 3 and Table 2.

In an alternative embodiment, the queries individually constituteindividual documents, and the scores, s(i, j) of Table 5 are producedfrom IR association matching functions. The table entries may thenrepresent measures of similarity between different documents. Amongother purposes, this information may be useful and valuable when thereis a natural ordering on, or grouping of, the documents/queries (e.g.,by time or other classification) and users wish to examine patterns thatarise across these natural orderings or groupings.

In Step 216, the user may filter the hits retrieved by reviewing thehits and determining whether the hit is reasonable in view of theidentified commodity or in view of predetermined parameters set for agiven search, e.g., industries that should not be considered. Forexample, if the identified commodity is aluminum, and a query of theproperties of aluminum was found in hits relating to diapers, and thediaper industry is one in which aluminum would never be used, or is anarea the requesting customer would never enter into, the user may filterout any hits that are related to diapers, thus narrowing the number ofhits to be reviewed. This could require reconsideration of any or all ofSteps 204, 206, 208, and 210.

Similarly to the ability to filter the retrieved documents, the user maybe able to expand the list of hits found by analyzing the hits-receivedand generating broader or narrower properties that were not consideredin the initial generation of properties in Step 204. Generation of thesealternate properties may then require the user to revisit any or all ofSteps 206, 208, and 210 to redefine the collection of documents tosearch and/or to conduct a search on a new set of queries and/or using anew IRS.

The search in Step 212 is then repeated using the revised premises. Asstated above, in the case of a narrowing revision, the file(s)containing the previous list of hits may be used as a selectedcollection of documents for the revised search.

Once the search in Step 212 is considered satisfactory, in Step 218 theuser or the automated process decides whether to add another dimensionto the array by further searching the existing array according to athird taxonomy or set of queries. Although a two-dimensional array iscommonly used for presentation of information to human users,three-dimensional arrays can be used where the amount of immediatelyvisible data in each cell is small, for example, by applying graphicaldisplay tools to render the array transparent, so that internal cellscan be seen, by highlighting selected cells or layers or the informationtherein, and so on. Arrays with any number of dimensions can bepresented by a system that displays only selected planes within thearray at any time.

In an embodiment, each cell in the array contains a visible and/ormachine-readable number indicating an IR score derived from a matchingfunction. Examples of such scores are the number of hits assigned tothat cell, and measures of the similarity or association of the hits tothe original query. In addition, a link to the data file or filescontaining those hits. Alternatively, or in addition, the cell maycontain a link to a file or report containing analysis data concerningthe hits.

In Step 210, upon review of the hits that are found in particular cellsof the array, possible applications are identified and defined. Thesepossible applications may relate to different uses of the identifiedcommodity that were not considered in the past. The applications may beidentified by examining the documents that are associated withidentified properties of the commodity, and fall within specific taxa ofthe classification, but are not associated with the actual commodity. Ifthese documents relate to a particular category or industry, then anassociation may be made between the commodity and the particularcategory. Returning again to the example discussed above, if the drugthalidomide (commodity) has a property of angiogenesis, and there aredocuments relating to angiogenesis that are not related to thalidomide,and some of the documents not related to thalidomide can be categorizedas being about some form of cancer, then an association may behypothesized that thalidomide may interact with cancer through theangiogenesis property.

Examination of the documents and determination of a possible newcommodity association with a category is preferably accomplished usinghuman judgment. Although it is preferable to use human judgment, thisexamination and determination may be automated. For example, anautomation method may rank order the properties identified in Step 204based on the strength of their association with the documents in aselected subclass; or rank order the documents in a selected subclassbased on the strength of their association with a selected property.Although two methods of automatically generating a possible opportunityhave been disclosed, other methods of generating possible meaningfulopportunities may be utilized and fall within the scope of the presentinvention.

The user, upon examining and determining any possible opportunities, maythen start the search method again utilizing properties that wereidentified from the method disclosed above and may or may not have beenconsidered before.

The search method and system of the present embodiment innovativelycombines principles and techniques from Information Retrieval andmatches commodity descriptions with potential applications andindustries.

Although the method of the present embodiment has been disclosed asgenerating possible novel applications or unknown properties of a givencommodity, as defined herein, alternatively, the method and system ofthe present embodiment may generate a commodity, as defined herein, froma given application, use or market. Returning again to the example usedhereinbefore, if an application is identified, such as cancer treatment,the method and system of the present embodiment returns unanticipated ornovel commodities, such as thalidomide, using properties or descriptivewords of the application.

In an initial stage of the examination, the user may inspect the arrayof cells for patterns or other phenomena visible in the data actuallydisplayed in the array, without yet drilling down to the underlyingdocuments themselves. For example, the user may look for cells withanomalously high or anomalously low numbers of hits, and especially forrows of cells where a taxon in the classification has anomalous numbersof hits relative to a high proportion of the queries of Step 210. To aidin visualization or other analysis, the number of hits in each cell maybe normalized to suppress variations in the total number of hits pertaxon or per query. Where an objective formula for identifyinginterestingly anomalous cells or rows is available or can be provided,the inspection may be automated. One of the queries may be the commodityitself, and taxa that have a high number of hits for the commodity maybe given a negative weighting if only novel uses for the commodity arebeing sought, on the assumption that taxa having a high number of hitsfor the commodity are likely to represent uses already known.

Where those rows apparently of interest tend to show a consistentvariation from one query to another, that may indicate that some queriesare better than others at discriminating the rows of interest, which mayassist in reformulating the queries at Step 216.

This embodiment discloses a method of providing information beyond thatwhich is generally associated with the subject commodity. Access to thisexpanded information allows the user to expand the scope of theusefulness of a commodity in order to find uses, applications, andmarkets for the commodity that were not, and would not have been,originally considered by the user. Alternatively, the present processallows the user to find commodities from a given use, application,market or industry that would not have been originally considered.

The present methods involve the interaction of three principal elements:taxonomies, information retrieval methods, and source documentcollections. These may be configured in a number of preferredembodiments.

Taxonomies

A taxonomy is a classification scheme, the elements of which are calledtaxa (singular: taxon). The concept is quite general:

Almost anything, including animate objects, inanimate objects, places,events, and activities may be classified according to some taxonomicscheme. Taxonomies are frequently hierarchical in structure. Howevertaxonomy may also refer to relationship schemes other than hierarchies,such as network structures. Other taxonomies may include single childrenwith multi-parents, for example, “Car” might appear with both parents“Vehicle” and “Steel Mechanisms.” A taxonomy might also be a simpleorganization of objects into groups, or even an alphabetical list.

The terms taxonomy, classification system or scheme, and categorizationsystem or scheme have this broad sense and are used in this broad way inthe present description. A number of related terms, some with morespecialized meaning but still instances of taxonomies in this broadsense, are in current use as terms of art for classifications that maybe used with various embodiments of the present invention. Theseinclude: ontology, thesaurus, vocabulary, controlled vocabulary,catchwords, meta-model, faceted index, faceted classification,folksonomy, and conceptual framework.

Examples of kinds of taxonomies (ontologies, controlled vocabularies,etc.) include: Product Classifications, such as the United NationsStandard Products and Services Code, the United Nations Central ProductClassification, or product catalog classifications by which vendors,portals, and other agents organize products for search an viewing bytheir customers and other users; Patent Classifications, such as theUnited States Patent Classification (USPC) or the International PatentClassification; Industry Classifications, such as NAICS (North AmericanIndustry Classification Scheme), SIC (United States Standard IndustrialClassifications), ISIC (International Standard Industrial Classificationof all Economic Activities), SITC3 (Standard International TradeClassification); activity classifications, such as WordNet (Verbrelationships), United States Bureau of Labor Statistics StandardOccupation Classification System (SOC); time/date classifications, suchas those provided by the ISO; location/place classifications, such as USGSA Geographic Locator Codes (US GSA GLC), United States DirectMarketing Areas (DMA); bibliographic, archival and museumclassifications using thesauruses and controlled vocabularies, such asthe Library of Congress Classification system (LoC), the Art andArchitecture Thesaurus, the Union List of Artist Names, the GettyThesaurus of Geographic Names (TGN), the UNESCO Thesaurus, the Cook'sThesaurus; education classifications, such as UK Joint Academic CodingSystem (JACS), UK Higher Education Standard Authority Coding (HESACODE);computing classifications, such as the ACM Computing ClassificationSystem; medical classifications such as ICD10 (InternationalClassification of Diagnoses), ICPC (International Classification ofPrimary Care), CPT (Current Procedural Terminology), US FDAClassification of Medical Devices; company classifications, such asFortune 500, S&P 500, INC 500.

In addition, the following are further examples of taxonomies that canbe used in embodiments of the present invention: the Linnaean taxonomy,for classifying living beings; the periodic table for classifying theelements; various product classification systems, e.g., as used forimport and export control; electronic dictionaries, such as WordNet;electronic encyclopedias, such as the Wikipedia; categorized electronicknowledge bases, such as Cyc (http://www.cyc.com/ andhttp://www.opencyc.org/); graphical taxonomies such as workflowdiagrams, flow charts, and organizational charts; Internationalclassifications such as the Nice Agreement concerning the InternationalClassification of Goods and Services for the Purposes of theRegistration of Marks, the International Classification for IndustrialDesigns under the Locarno Agreement, the International Classification ofthe Figurative Elements of Marks under the Vienna Agreement; standardsspecifications, such as MILSPEC; standards from the InternationalOrganization for Standardization (ISO); and industry, national,international, and United Nations standards generally.

As an example, in the LoC, at the highest level after the root of thetree, the nodes in the are labeled with single letters of the alphabet,e.g., A for “General Works,” B for “Philosophy, Psychology, Religion,” Tfor “Technology,” and so on. Single letters followed by single lettersconstitute the next more specific level, e.g., BC for “Logic” and BF for“Psychology.” After that, a numbering scheme is used, e.g. BC 11-39 isfor the history of logic. The scheme continues for several levels.

Because different classification schemes have widely varying formats, afile may be created that captures a particular classification scheme andmaps the nodes of the particular classification scheme to standard NodeID notations that can be used for generation of a mapping in accordancewith embodiments of the present systems and methods.

Another example of a preexisting classification system, or taxonomy, isthe United Nations Standard Products and Services Code (UNSPSC), aportion of which is shown in Table 6.

TABLE 6 EGCI Segment Family Class Commodity Title 008991 10 00 00 00Live Plant and Animal Material and Accessories and Supplies 009044 10 1000 00 Live animals 009339 10 10 15 00 Livestock 000001 10 10 15 01 Cats000002 10 10 15 02 Dogs 000003 10 10 15 04 Mink 000004 10 10 15 05 Rats000005 10 10 15 06 Horses 000006 10 10 15 07 Sheep 000007 10 10 15 08Goats 000008 10 10 15 09 Asses 000009 10 10 15 10 Mice 000010 10 10 1511 Swine 000011 10 10 15 12 Rabbits 000012 10 10 15 13 Guinea pigs000013 10 10 15 14 Primates 000014 10 10 15 15 Armadillos 000015 10 1015 16 Cattle 010976 10 10 15 17 Camels 009340 10 10 16 00 Birds and fowl000016 10 10 16 01 Live chickens 000017 10 10 16 02 Live ducks 000018 1010 16 03 Live turkeys 000019 10 10 16 04 Live geese 000020 10 10 16 05Live pheasants 009341 10 10 17 00 Live fish 000021 10 10 17 01 Livesalmon 000022 10 10 17 02 Live trout 000023 10 10 17 03 Live tilapia014985 10 10 17 04 Live carp 014986 10 10 17 05 Live eels

As is evident from Table 6, this is a hierarchical classification systemof depth 4: segments, families, classes, and (as leaves) commodities,which latter are identified by having non-00 numbers in the Commoditycolumn of the table in the Figure. The names of the taxa (whether or notthey are leaves in the classification system) may be either singleterms, e.g., “cats” and “dogs,” or they may consist of two or moreterms, e.g., “live animals” and “birds and fowl.” Following standardusage in this field, we say that the taxa are described with n-grams,word phrases of 1 or more terms, where n is the number of terms. Thus,“cats” is a 1-gram, “live animals” is a 2-gram (or bi-gram), “birds andfowl” is a 3-gram (or tri-gram), and so on generally. A particular taxonmay be more naturally interpreted not as a pure n-gram but as a Booleancombination of n-grams. “Birds and fowl” in Table 6 constitutes anexample. Reinterpretation or re-expression of taxa literals (e.g.,“birds and fowl”) lies within the scope of the present invention. Forexample, by policy expressions of the form “P and Q” might bere-expressed for purposes of inputting to an IRS as “P AND Q” or “P ORQ,” depending on context and current purposes.

Although exemplary classification schemes have been disclosed, any otherclassification scheme may be utilized as well. Where the search in Step210 is based on a classification scheme, that scheme is not limited topre-existing classification schemes. Classification schemes generated byhuman intuition, based on the hits that have been received or theproperties that have been used, for example, may also be utilized by theuser. The present invention does not limit the user to selecting oneclassification scheme, multiple classification schemes may be utilizedat the user's discretion.

A portion of a classification scheme, consisting of two or more taxa, isalso a classification scheme or taxonomy.

Information Retrieval Methods

The purpose of an Information Retrieval System (IRS) is to retrieve asmany as possible of the documents, D, in a collection, C, relevant to aquery, Q, at the same time retrieving as few of the non-relevantdocuments as possible. For many purposes certain types of unreliabilitymay be less harmful than other types. In particular, false positives arefrequently less undesirable than false negatives, because a falsenegative is effectively a lost hit, whereas false positives can bescreened out in the course of a review of the hits. Examples of purposeswhere false positives are usually preferred include environmentalscanning and business intelligence. When used for such purposes, the IRSmay therefore be configured to retrieve all or substantially all of therelevant documents, even at the expense of retrieving more non-relevantdocuments.

IR systems need one or more matching functions by which they identifyrelevant and non-relevant documents. There are very many such matchingfunctions commonly in use and the present process may avail itself ofany that prove useful, including, but not limited to, a number ofdifferent ways of implementing the matching function that are already inuse. Some of these different approaches may be classified, at least inpart, as follows.

1. String matching views a query as a (character) string (sequence ofcharacters). A determination is made of whether, and if so how often,the string occurs in each of the documents in a given collection. For agiven document D in collection C and query Q, the string match functionm(Q,D) returns a 0 if the string is not present in the document. If thestring is present, the match function may return either a logical 1 or acount of the number of occurrences of the string in the document,depending on the implementation. Further, partial or pattern-basedmatches are permitted, such as matching the query “color” to both“color” and “colour.” This is often achieved with various forms of“wild-card” searching and with regular expression matching. “Stemming,”in which a word is shortened to its core part to simplify matching toinflected and derivative forms, may also be used. “Stopping,” in whichcertain very common words that have little content and that do nottypically serve to distinguish documents, such as “the,” “and,” and thelike, may also be used. Stemming, stopping, and similar techniques,although most commonly applied to string matching, can be applied toother forms of search query that are based on words.

2. Boolean matching extends string matching to allow logicalcombinations of search strings in a query. Thus, for example, the query“red OR blue” would match to any document containing either the string“red” or the string “blue” or both. OR is called disjunction and is oneof a number of what are called Boolean operators or logical connectives.Conventionally, the term “Boolean matching” includes use of non-logical,non-Boolean operators, such as proximity operators. For example, a querymay request matching of “big” appearing within 3 words before “circus.”As in the case of string matching, a Boolean matching function willreturn a 0 for a document if the specified combination of strings is notmatched in the document. Similarly, if the combination is present, thematch function may return either a 1 or a count of the number ofoccurrences of the combination in the document, depending on theimplementation. Further, partial or pattern-based matches are permitted.

3. An association matching function m(Q,D), for a given document D incollection C and query Q, returns a number (or possibly a collection ofnumbers) measuring an association between Q and D; typically this is inthe form of a score measuring either similarity or dissimilarity or inthe form of a rank on the document in the collection. Associationmatching functions may be either deterministic or probabilistic, thelatter typically returning a score that indicates a probability of adocument's being relevant to the query. Association matching functionspresently in use include in various forms Dice's coefficient, Jaccard'scoefficient, the cosine coefficient, the overlap coefficient, and latentsemantic indexing.

Association matching functions typically are able to handle largenumbers of individual query terms. In consequence, a query may be takento be as extensive and complex as an entire document. Thus, associationmatching functions typically are able to produce an association measurebetween two documents. In effect, document D0 replaces Q and thematching function may be written m(D0,D). Thus, a document may be usedas a query.

For the purposes of this embodiment, any matching function such as thosecommonly in use in Information Retrieval may potentially be used, thedecision being made on the basis of practical considerations as well asperformance and accuracy considerations.

Various embodiments of the present invention can use established,well-known information retrieval methods in order to match taxa fromclassification schemes to documents for the purpose of classifying thedocuments. A large class of information retrieval methods usable inembodiments of the present invention may be specified by two aspects:document representation; and document matching.

A document may be represented in a variety of ways, including wayscommon in the art of information retrieval, including: by its originalform, by a processed form that largely preserves the original words butremoves markup and other extraneous elements, and by an inverted index(term-document file or table) that maps a term (word, phrase, iconicelement) to a document and/or a position in a document. Other forms ofdocument representation, many of them more complex, are known in the artof Information Retrieval, or may hereafter be developed, and areincluded among the ways in which documents may be represented for thepresent purpose.

Matching of taxa, or taxa as augmented with additional search terms asdescribed above, to documents may proceed in any of a variety of ways,including ways already known in the art of information retrieval,including: direct string matching (as literals or as patterns, e.g. withwildcards or conforming to a regular expression), direct Booleanmatching, various functional transformations such as tf−idf (termfrequency minus inverse document frequency) weighting, cosine measure,various coefficients of association, including but not limited toJaccard's coefficient, and such known methods as latent semanticindexing and methods related to it.

In short, the present embodiment encompasses using any suitableinformation retrieval technique, including known information retrievaltechniques, primed by a taxonomy or classification scheme, to categorizea document collection by matching individual taxa with documents, whichmatching may be full (binary, matched or not) or by degree (partialmatching, degree of matching). Further, the matching may be done in anyof a number of ways, including literal presence of a query term (e.g.,the search string is “live cats” and the retrieval system matches it todocuments containing the literal string “live cats”), pronunciation(e.g., the search string is “colour” and the retrieval system matches itto documents containing words pronounced similarly, such as “color” and“colour,” using, e.g., the Soundex algorithm), and in the case oficonographic information shape, color, or other visible distinguishingcharacteristics.

“Direct association” is the direct mapping of a hit with aclassification subclass using information included with the hit. Forexample, an exemplary hit, DOC 1, is a patent. As is known to thosefamiliar with patents, each invention is classified by the Patent Officeinto particular classes and sub-classes. Depending on the invention, thePatent Office may classify the invention into a plurality of classes andsubclasses. If the user has selected to use the USPTO classificationscheme, a direct association of DOC 1 to the classes and subclasses ofthe USPTO classification subclasses may be conducted using the class andsubclass numbers associated with DOC 1 by the Patent Office. In apreferred embodiment, direct association is conducted automaticallyusing the selected classification scheme and a classification node file.

Another example would be to utilize, for example, the LoC classificationscheme and associate a hit comprising a LoC call number to the subclassassociated with the call number in the LoC classification scheme. Anyhit that comprises a classification number of a pre-existingclassification scheme can be directly associated with the selectedclassification scheme using the classification number included with thehit.

“Indirect association” is the association of a hit to a selectedclassification scheme subclass based on information other than aclassification number directly related to the classification scheme.Indirect association may utilize imperfect matching, which uses humanjudgment as a means of associating a hit with a subclass. Imperfectmatching may also use association of information included with the hitthat is not able to be directly associated with a subclass, e.g., anISSN number. An example of a method of imperfect matching uses a sampleof the hits that have been retrieved and, using human judgment,associates a hit with a subclass of the selected classification scheme.Then all remaining hits that are similar to the associated hit are alsoassociated with the subclass.

The association of information included with the hit may be utilized ina similar manner as the above disclosed method. A relationship betweenthe information included with the hit and a subclass of the selectedclassification scheme is determined using table algorithms that considerthe subject matter of the hits and the information that may beassociated with the subclasses of the classification scheme. As stated,an example is an ISSN included with a hit. The present processdetermines an association between the ISSN and a subclass of theselected classification scheme. For example, suppose that a “hit” citesan article in the journal Communications of the ACM. The ISSN number is0001-0782, and would be present in the hit record, typically. Thepresent process extracts the ISSN number from the hit record (by patternmatching on text) and checks to see if that ISSN number exists in theLoC cataloging records.

If the ISSN exists, the system looks up the ISSN, finds thecorresponding LoC catalog entry, extracts the LoC classification (taxon)from that catalog entry, and uses the result to map the original articleto the specified taxon in an LoC classification file. If the ISSN is notmatched to the LoC catalog, an exception report is generated by thesystem and a human or automated process looks up the LoC call number forthat ISSN and/or serial title.

Example Sources of Source Documents

The third principal element in the present system and process is acollection of source documents, which is to be categorized by ataxonomy. Such collections may be obtained in many different ways andfrom many different sources, including the following: all of thedocuments on the World Wide Web that are indexed by a search engine,such as Google, Yahoo! etc.; the documents returned by, or identifiedby, a search or retrieval engine in response to a query request;documents posted at the Web site of an organization; documents posted onthe World Wide Web and falling into a standard category, such as blogs,blogs on a general topic, and wikis; published patents or patentapplications; regulatory filings, such as SEC documents (10-K, 10-Q,etc.), safety filings, environmental reports, political campaignfilings, etc.; legal and administrative records; archives, or portionsof archives, maintained by organizations; collections of e-mailmessages, SMS text messages, and the like; digital libraries andcollections, such as those maintained by the National Library ofMedicine; repositories of technical reports, such as those maintained byorganizations such as NIST (National Institute of Standards andTechnology), universities, and research organizations; archives ofnewspapers and other periodicals; commercially marketed documentcollections, such as provided by Factiva, Dialog and LexisNexis;scientific, engineering, professional, or commercial collections, suchas Medline and the National Institute of Standards and Technology;standards specifications (including performance specifications)containing substantial amounts of descriptive narrative and/or drawingsand illustrations, such as MILSPEC, ISO (International Organization forStandardization) documents, and United Nations standards.

Also, any combination of the above may constitute a document collectionfor the present purpose.

Similarly, it is often useful and valuable, or even required, to be ableto search a body of information, collected as records, for the purposeof discovering new information, information that is not contained in anysingle record of the repository but that resides in, or is apparentfrom, or emerges from multiple records in the repository. For example, afirm's buying and selling transactions may be stored as records in arelational database system. No single record (of a purchase or a sale)contains the information of whether sales are increasing or decreasing.This information is, however, available by processing multiple records(sales records over a period of time), classifying (e.g., by month) andaggregating (e.g., by summing quantities sold) the information in them.Queries done in these ways-whether directed at data records in arepository or document records in a repository-are thus said to be“pattern-oriented.”

Certain elementary pattern-oriented queries may be undertakensuccessfully with record-oriented technology (e.g., SQL, standard IRsystems, including Internet search engines). However, record-orientedtechnology is generally inadequate to the task of processingnon-elementary pattern-oriented queries. In consequence, the field ofdata mining or KDD (knowledge discovery in databases, aka: knowledgediscovery and data mining) has arisen to create knowledge and supply themarket with products for supporting pattern-oriented queries inrepositories of data records. Similarly, the field of text mining ortext data mining has arisen to create knowledge and supply the marketwith products for supporting pattern-oriented queries in repositories oftext records.

As may be seen from the foregoing description, embodiments of thepresent invention are broadly useful for the purpose of text mining. Thepresent embodiment uses IR systems, including but not limited tostandard IR systems, concepts, and technologies in conjunction withexisting classification systems (a) to classify collections of documentrecords and (b) to assist in producing numerical statistics for theclassified documents, which may be aggregated in various ways and usedfor decision making.

Information from CDBs produced by the present embodiments, and indeedtext mining information in general, will often be noisier and lessreliable than information produced by data mining, because data miningcan often be based on very precise and reliable data. Nevertheless, textmining information produced with reliance on embodiments of the presentinvention will often be quite useful and valuable for decision makingbecause required information is not otherwise available; in particular,because data records are not available that produce a better result.Further, for many purposes the breadth of information afforded bytextual sources is an important, even essential, asset.

Thus, text mining can often be very useful for such areas and purposesas market research, finding new uses for known commodities, findingcommodities that meet given requirements, environmental scanning ormonitoring, business intelligence, scenario planning, and landscaping(in the business strategy sense), and investment analysis.

While the foregoing specification has been described with regard tocertain preferred embodiments, and many details have been set forth forthe purpose of illustration, it will be apparent to those skilled in theart without departing from the spirit and scope of the invention, thatthe invention may be subject to various modifications and additionalembodiments, and that certain of the details described herein can bevaried considerably without departing from the basic principles of theinvention.

For example, when generating hit files from machine-readable sourcedocuments, it is possible to “turn text into data.” A datum, or anindicator, is essentially a variable given a numerical value. E.g.,variable=height, value=2 meters. To be really valuable, the data, orindicators, need to be put into categories, so a user can discoveruseful patterns. In the present example, the variable “height,” asapplied to people, might be categorized by gender, age, ethnic heritage,economic status and so forth. When categorizing documents, it ispossible to count various things—words, phrases, number of documents,etc.—and that generates numbers, some of which may be useful to a user.

The numerical data thus extracted can be reported out in tables, fed tovisualization routines for graphic display, included in analysis reportssuch as those mentioned above as associated with cells in an array, andso on. Reports may be presented to a human user, or further analyzedelectronically, or both.

Where the description refers specifically to the use of a “searchstring” as a query, other forms of query may be substituted.

In the interests of simplicity, embodiments of the invention have beendescribed as generating and using two-dimensional arrays or tables. Thearrays or tables may be extended to three or more dimensions, even wherethat is not explicitly stated.

Especially in a mode using a query set for at least one dimension of thearray, there will frequently be a substantial number of documents thatdo not return a “hit” for any Q_(j). Depending on the intended use ofthe information, these documents may be ignored, or a column in thearray may be assigned for no-hit documents. In a multi-dimensional arrayusing two or more query sets, it may be desirable to assign cells in thearray to documents that return hits for some query sets but not forother query sets. The cell for documents that return no hits at all, andoptionally also some other cells, for example, cells for documents thatreturn fewer than a threshold number of hits, may be void cells in whichno data are entered. Any marginals for a row or column containing a voidcell may then be defined specially to compensate appropriately for thepresence of void cells.

In a classification-classification mode, it is more likely that alldocuments will be classified, except documents outside the scope of theclassification, which will typically be ignored. However, both inclassification-query mode and in classification-classification mode,where unclassified documents do exist under any classification beingused, they may be treated in any of the ways mentioned above fordocuments that return no hit to a query set.

Thus, it is intended that the present invention cover the modificationsand variations of this invention provided they come within the scope ofthe appended claims and their equivalents.

1. A computer-based method for determining patterns of information in atleast one categorized collection, the method comprising: acquiringcategorized information comprising at least one source collectionconsisting of documents, retrievable information other than documents,and combinations thereof, classified with at least a first taxa-basedclassification scheme; acquiring an Information Retrieval (IR) engineand at least one IR query; applying the at least one IR query and the IRretrieval engine for assigning numerical scores indicating relevance ofeach document, retrievable information other than documents, orcombination thereof for each category or set of categories in the atleast first taxa based classification scheme; mapping the numericalscores with regard to the at least first taxa-based classificationscheme, and creating therefrom a numerical mapping record, therebyassociating the category or categories in the taxa-based classificationscheme to the at least one IR query; examining the numerical mappingrecord for presence of one or more patterns of numerical scores; anddisplaying a report comprising the patterns of numerical scores.
 2. Themethod according to claim 1, further comprising generating at least onehuman-readable image of an array of the numerical scores, or ofinformation generated by analysis of the array.
 3. The method accordingto claim 1, further comprising providing the generated at least onearray image or the information generated by analysis of the array, to acustomer in commerce.
 4. The method according to claim 1, furthercomprising: providing at least a second taxa-based classificationscheme; selecting one or more IR queries and applying same to thecategorized information for each category or set of categories in the atleast second taxa based classification scheme; applying the one or moreIR queries for assigning numerical scores indicating relevance for eachdocument, retrievable information other than documents, or combinationthereof, for the at least second category or second set of categories inthe categorized information; comparing or combining the numerical datafrom the applied first category or first set of categories to the atleast second category or second set of categories defined with referenceto the one or more second IR queries; mapping same and generating atleast one array of numerical data relating to documents, retrievableinformation other than documents, or combination thereof, wherein the atleast first category or first set of categories and the at least secondcategory or second set of categories provide axes of the array; andexamining the numerical mapping record for presence of one or morepatterns of numerical scores.
 5. The method according to claim 4,wherein the IR queries describe a property or combination(s) ofproperties of a commodity, and wherein querying comprises using IRmethods and algorithms in conjunction with query terms and patterns ofquery terms and phrases used to describe the property or combinations ofproperties.
 6. The method according to claim 5, further comprisingcomparing the numerical data relating to documents, retrievableinformation other than documents, or combination thereof in differenttaxa, and reviewing taxa having higher numerical scores.
 7. The methodaccording to claim 6, wherein generating an array comprises generating ahuman-readable image of the array in which a cell displays the numericalscores associated with each taxon of the at least first or at leastsecond taxa-based categorization scheme, and each category thatintersect at the cell.
 8. The method according to claim 7, furthercomprising providing at least one of the array image, or of informationgenerated by analysis of the array, or combinations thereof, to acustomer in commerce.
 9. The method according to claim 7, comprisingcreating at least one categorized collection, wherein providing thedocuments, retrievable information other than documents, or acombination thereof, assigned to the at least second category or atleast second set of categories, comprises automatically classifying thedocuments, retrievable information other than documents, or acombination thereof, in the source collection using the IR engine inconjunction with the taxa-based classification scheme(s) to assign oneor more of the documents, retrievable information other than documents,or a combination thereof, to the at least one category or set ofcategories.
 10. The method according to claim 9, wherein assigning oneor more of the documents, retrievable information other than documents,or a combination thereof to at least a second category or at least asecond set of categories further comprises selecting one or morequeries; selecting an IR system; querying the documents using the IRsystem according to the one or more queries; and assigning one or moreof the documents, retrievable information other than documents, or acombination thereof, to a plurality of categories defined with referenceto the one or more queries.
 11. The method according to claim 9, furthercomprising generating a numerical array comprising a human-readablearray image in which a cell displays the numerical scores associatedwith each taxon of the at least first or at least second taxa-basedcategorization schemes, or combination thereof, and each category thatintersect at the cell.
 12. The computer-based method for determiningpatterns of information in accordance with claim 1, further comprisingproviding additional taxa-based classification schemes numbering n;selecting one or more IR queries and applying same to the categorizedinformation for each category or set of categories in the n taxa basedclassification schemes; assigning numerical scores indicating relevancefor each document, retrievable information other than documents, or acombination thereof, for each category or sets of categories n in thecategorized information; assigning numerical scores indicating relevancefor the categories or sets of categories defined with reference to theIR queries; mapping same and generating at least one array of numericaldata indicating relevance of each document, retrievable informationother than documents, or combination thereof, wherein the categoriesprovide axes of the array; and examining the numerical mapping recordfor presence of one or more patterns of numerical scores.
 13. The methodaccording to claim 12, wherein the IR queries describe a property orcombination(s) of properties of a commodity, and wherein queryingcomprises using IR methods and algorithms in conjunction with queryterms and patterns of query terms and phrases used to describe theproperty or combinations of properties, and wherein generating the arraycomprises generating a human-readable image of the array in which a celldisplays the numerical scores associated with each taxon of categoriesthat intersect at the cell.
 14. The method according to claim 12,further comprising comparing the numerical data relating to documents,retrievable information other than documents, or combination thereof indifferent taxa, and reviewing taxa having higher numerical scores. 15.The method according to claim 14, further comprising providing at leastone of the array image, or of information generated by analysis of thearray, or, combinations thereof, to a customer in commerce.
 16. Themethod according to claim 1, comprising creating at least onecategorized collection, wherein providing the documents, retrievableinformation other than documents, or a combination thereof, assigned tothe at least a one category or at least one set of categories, comprisesautomatically classifying the documents, retrievable information otherthan documents, or a combination thereof, in the source collection usingthe IR engine in conjunction with the taxa-based classificationscheme(s) to assign one or more of the documents, retrievableinformation other than documents, or a combination thereof, to the atleast one category or set of categories.
 17. The method according toclaim 16, comprising creating at least n categorized collections,wherein providing the documents, retrievable information other thandocuments, or a combination thereof, assigned to the at least ncategories or sets of categories, comprises automatically classifyingthe documents, retrievable information other than documents, or acombination thereof, in the source collection using the IR engine inconjunction with the taxa-based classification scheme(s) to assign oneor more of the documents, retrievable information other than documents,or a combination thereof, to the at least n category or categorieswithin the set.
 18. The method according to claim 17, wherein assigningone or more of the documents, things other than documents, or acombination thereof to n number of categories or set of categoriesfurther comprises selecting one or more queries; selecting an IR system;querying the documents using the IR system according to the one or morequeries; and assigning one or more of the documents, retrievableinformation other than documents, or a combination thereof, to aplurality of categories defined with reference to the one or morequeries.
 19. The method according to claim 17, further comprisinggenerating a numerical array comprising a human-readable array image inwhich a cell displays the numerical scores associated with each taxon ofthe first or at least second taxa-based categorization scheme, orcombinations thereof, and each category that intersect at the cell. 20.A computer-based, information management system for determining patternsof information in at least one categorized collection, the systemoperably comprising: memory processing receiver for acquiringcategorized information comprising at least one source collection ofdocuments, retrievable information other than documents, or acombination thereof, classified with at least one taxa-basedclassification scheme; memory processing receiver for acquiring anInformation Retrieval (IR) engine and at least one IR query into thecomputerized system, for assigning numerical scores to indicaterelevance of the categorized information and map the numerical scores,creating; a numerical mapping record, thereby associating the categoryor categories in the taxa-based classification scheme to the at leastone IR query; processor for examining the numerical mapping record forpresence of one or more patterns of numerical scores; and reportercomponent for reporting patterns of numerical scores.
 21. The systemaccording to claim 20, comprising a computing device causing a programto run, thereby causing the computing device to carry out the receiving,assessing, and assigning steps.
 22. The system of claim 20, furthercomprising a device to identify and report documents corresponding tothe patterns of numerical scores in the report.
 23. The system of claim20, further comprising a device to receive at least a second category orsecond set of categories.
 24. The system according to claim 23, whereina computing device comprises causing a program to run, thereby causingthe computing device to carry out the receiving, assessing, andassigning steps.
 25. The system of claim 23, further comprising a deviceto identify and report documents corresponding to the patterns ofnumerical scores in the report.
 26. The system of claim 20, furthercomprising a device to receive at least n number of categories or set ofcategories.
 27. The system according to claim 26, wherein a computingdevice comprises causing a program to run, thereby causing the computingdevice to carry out the receiving, assessing, and assigning steps. 28.The system of claim 26, further comprising a device to identify andreport documents corresponding to the patterns of numerical scores inthe report.
 29. A computer-operated software program for determiningpatterns of information in at least one categorized collection ofdocuments, the program embodied in a non-transitory machine-readablemedium which, when running on a computer causes the computer to: receivean acquired categorized information comprising at least one sourcecollection of documents, retrievable information other than documents,or combinations thereof, classified with at least one taxa-basedclassification scheme; acquire an Information Retrieval (IR) engine andat least one IR query; apply the at least one IR query and the IRretrieval engine to assign numerical scores indicating relevance of eachdocument, retrievable information other than documents, or combinationsthereof, for each category in the at least one taxa based classificationscheme, or for at least one set of categories in the categorizedinformation; map the numerical scores with regard to the at least onetaxa-based classification scheme for the at least one category or set ofcategories, and creating therefrom a numerical mapping record, therebyassociating the category or categories in the taxa-based classificationscheme to the at least one IR query; examine the numerical mappingrecord for presence of one or more patterns of numerical scores; anddisplay a report comprising the patterns of numerical scores.
 30. Thecomputer-based software program of claim 29, further comprising causingthe computer to identify and report documents, retrievable informationother than documents, or a combination thereof, corresponding to thepatterns of numerical scores in the report.
 31. The computer-basedsoftware program embodied in a machine-readable medium which, whenrunning on a computing system in accordance with claim 29, furthercauses the computing system to: receive at least one second category orsecond set of categories, one or more of the documents, retrievableinformation other than documents, or a combination thereof, beingassigned to one or more category or categories within each set; applythe IR system; generate at least one array of numerical data relating todocuments, retrievable information other than documents, or combinationthereof, wherein the categories provide axes of the array; and createtherefrom a numerical mapping record from which patterns are determined.32. The computer-based software program embodied in a machine-readablemedium of claim 31, further comprising causing the computer to identifyand report documents corresponding to the patterns of numerical scoresin the report.
 33. The software program for determining patterns ofinformation of claim 29, further comprising causing the computer toreceive acquired categorized information comprising at least n number ofcategories or n number of sets of categories, and applying the IR systemand mapping the documents, retrievable information other than documents,or a combination thereof, to n categories or sets of categories, andcreate therefrom a numerical mapping record from which patterns aredetermined.
 34. The computer-based software program of claim 33, furthercomprising causing the computer to identify and report documents,retrievable information other than documents, or a combination thereof,corresponding to the patterns of numerical scores in the report.